12 research outputs found
High-fidelity metaprogramming with separator syntax trees
Many metaprogramming tasks, such as refactorings, automated bug fixing, or large-scale software renovation, require high-fidelity source code transformations-transformations which preserve comments and layout as much as possible. Abstract syntax trees (ASTs) typically abstract from such details, and hence would require pretty printing, destroying the original program layout. Concrete syntax trees (CSTs) preserve all layout information, but transformation systems or parsers that support CSTs are rare and can be cumbersome to use. In this paper we present separator syntax trees (SSTs), a lightweight syntax tree format, that sits between AST and CSTs, in terms of the amount of information they preserve. SSTs extend ASTs by recording textual layout information separating AST nodes. This information can be used to reconstruct the textual code after parsing, but can largely be ignored when implementing high-fidelity transformations. We have implemented SSTs in Rascal, and show how it enables the concise definition of high-fidelity source code transformations using a simple refactoring for C++
Concrete Syntax with Black Box Parsers
Context: Meta programming consists for a large part of matching, analyzing,
and transforming syntax trees. Many meta programming systems process abstract
syntax trees, but this requires intimate knowledge of the structure of the data
type describing the abstract syntax. As a result, meta programming is
error-prone, and meta programs are not resilient to evolution of the structure
of such ASTs, requiring invasive, fault-prone change to these programs.
Inquiry: Concrete syntax patterns alleviate this problem by allowing the meta
programmer to match and create syntax trees using the actual syntax of the
object language. Systems supporting concrete syntax patterns, however, require
a concrete grammar of the object language in their own formalism. Creating such
grammars is a costly and error-prone process, especially for realistic
languages such as Java and C++. Approach: In this paper we present Concretely,
a technique to extend meta programming systems with pluggable concrete syntax
patterns, based on external, black box parsers. We illustrate Concretely in the
context of Rascal, an open-source meta programming system and language
workbench, and show how to reuse existing parsers for Java, JavaScript, and
C++. Furthermore, we propose Tympanic, a DSL to declaratively map external AST
structures to Rascal's internal data structures. Tympanic allows implementors
of Concretely to solve the impedance mismatch between object-oriented class
hierarchies in Java and Rascal's algebraic data types. Both the algebraic data
type and AST marshalling code is automatically generated. Knowledge: The
conceptual architecture of Concretely and Tympanic supports the reuse of
pre-existing, external parsers, and their AST representation in meta
programming systems that feature concrete syntax patterns for matching and
constructing syntax trees. As such this opens up concrete syntax pattern
matching for a host of realistic languages for which writing a grammar from
scratch is time consuming and error-prone, but for which industry-strength
parsers exist in the wild. Grounding: We evaluate Concretely in terms of source
lines of code (SLOC), relative to the size of the AST data type and marshalling
code. We show that for real programming languages such as C++ and Java, adding
support for concrete syntax patterns takes an effort only in the order of
dozens of SLOC. Similarly, we evaluate Tympanic in terms of SLOC, showing an
order of magnitude of reduction in SLOC compared to manual implementation of
the AST data types and marshalling code. Importance: Meta programming has
applications in reverse engineering, reengineering, source code analysis,
static analysis, software renovation, domain-specific language engineering, and
many others. Processing of syntax trees is central to all of these tasks.
Concrete syntax patterns improve the practice of constructing meta programs.
The combination of Concretely and Tympanic has the potential to make concrete
syntax patterns available with very little effort, thereby improving and
promoting the application of meta programming in the general software
engineering context
High-fidelity metaprogramming with separator syntax trees
Many metaprogramming tasks, such as refactorings, automated bug fixing, or large-scale software renovation, require high-fidelity source code transformations-transformations which preserve comments and layout as much as possible. Abstract syntax trees (ASTs) typically abstract from such details, and hence would require pretty printing, destroying the original program layout. Concrete syntax trees (CSTs) preserve all layout information, but transformation systems or parsers that support CSTs are rare and can be cumbersome to use. In this paper we present separator syntax trees (SSTs), a lightweight syntax tree format, that sits between AST and CSTs, in terms of the amount of information they preserve. SSTs extend ASTs by recording textual layout information separating AST nodes. This information can be used to reconstruct the textual code after parsing, but can largely be ignored when implementing high-fidelity transformations. We have implemented SSTs in Rascal, and show how it enables the concise definition of high-fidelity source code transformations using a simple refactoring for C++
Large-scale semi-automated migration of legacy C/C++ test code
This is an industrial experience report on a large semi-automated migration of legacy test code in C and C++. The particular migration was enabled by automating most of the maintenance steps. Without automation this particular large-scale migration would not have been conducted, due to the risks involved in manual maintenance (risk of introducing errors, risk of unexpected rework, and loss of productivity). We describe and evaluate the method of automation we used on this real-world case. The benefits were that by automating analysis, we could make sure that we understand all the relevant details for the envisioned maintenance, without having to manually read and check our theories. Furthermore, by automating transformations we could reiterate and improve over complex and large scale source code updates, until they were “just right.” The drawbacks were that, first, we have had to learn new metaprogramming skills. Second, our automation scripts are not readily reusable for other contexts; they were necessarily developed for this ad-hoc maintenance task. Our analysis shows that automated software maintenance as compared to the (hypothetical) manual alternative method seems to be better both in terms of avoiding mistakes and avoiding rework because of such mistakes. It seems that necessary and beneficial source code maintenance need not to be avoided, if software engineers are enabled to create bespoke (and ad-hoc) analysis and transformation tools to support it
usethesource/rascal-language-servers:
An LSP server for Rascal which includes an easy-to-use LSP generator for languages implemented in Rascal, and an interactive terminal REPL
Large-scale semi-automated migration of legacy C/C++ test code
This is an industrial experience report on a large semi-automated migration of legacy test code in C and C++. The particular migration was enabled by automating most of the maintenance steps. Without automation this particular large-scale migration would not have been conducted, due to the risks involved in manual maintenance (risk of introducing errors, risk of unexpected rework, and loss of productivity). We describe and evaluate the method of automation we used on this real-world case. The benefits were that by automating analysis, we could make sure that we understand all the relevant details for the envisioned maintenance, without having to manually read and check our theories. Furthermore, by automating transformations we could reiterate and improve over complex and large scale source code updates, until they were “just right.” The drawbacks were that, first, we have had to learn new metaprogramming skills. Second, our automation scripts are not readily reusable for other contexts; they were necessarily developed for this ad-hoc maintenance task. Our analysis shows that automated software maintenance as compared to the (hypothetical) manual alternative method seems to be better both in terms of avoiding mistakes and avoiding rework because of such mistakes. It seems that necessary and beneficial source code maintenance need not to be avoided, if software engineers are enabled to create bespoke (and ad-hoc) analysis and transformation tools to support it
usethesource/vallang: v0.14.4
Replacing custom threads with jobs on the global JVM threadpool
Fixed #119 to get rid of the reflection warning prints on java 1